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Why GAO Did This Study 

The Youth Opportunity Grant 
program (YO) represented an 
innovative approach to improving 
education and employment 
opportunities for at-risk youth by 
targeting resources in high poverty 
areas and incorporating strategies 
that experts have identified as 
effective for serving this 
population. The Department of 
Labor (the Department) awarded 
36 grants in 2000, and the program 
continued for 5 years. The 
Department had used a similar 
approach on a smaller scale in 
previous programs, but little 
information is available on the 
impact of these other programs. In 
order to understand what can be 
learned from the Youth 
Opportunity Grant program, GAO 
examined the grantees’ 
implementation of the program, 
challenges they faced, and what is 
known about the program’s 
outcomes and impact. To view 
selected results from GAO’s Web- 
based survey of the Program 
Directors, go to GAO-06-56SP. 



What GAO Recommends 



GAO recommends that the 
Secretary of Labor take the actions 
necessary to complete the impact 
analysis of the Youth Opportunity 
Grant program and release the data 
and all related research reports 
from the program’s evaluation. In 
comments on GAO’s draft report, 
the Department agreed with GAO’s 
recommendation to complete the 
impact analysis and publish all 
related reports. 



www.gao.gov/cgi-bin/getrpt7GAO-06-53. 

To view the full product, including the scope 
and methodology, click on the link above. 

For more information, contact David Beilis at 
(415) 904-2272 or bellisd@gao.gov. 



What GAO Found 

Grantees used a variety of approaches to build the infrastructure of the YO 
program, provide services to at-risk youth, and conduct outreach efforts. 
While grantees set up centers and trained core staff to deliver services, they 
differed somewhat in their approaches, depending on circumstances within 
their communities. In addition, grantees employed a combination of 
strategies to provide youth services, including collaborating with other 
providers and inventing unique programming. To recruit this hard-to-serve 
target population, grantees used a range of techniques, from partnering with 
juvenile justice agencies, to combing malls for eligible youth. 

Fast program start up, conditions in their communities, and the 
characteristics and needs of the youth challenged the grantees; however they 
used features of the program design to address some of these difficulties. 
Many grantees struggled to set up the program, especially within the 
Department’s time frame. In addition, grantees felt encumbered by the drugs, 
violence, and a lack of jobs in their communities as well as the obstacles 
facing their clients, such as low academic achievement and lack of family 
support. Grantees used the discretion and other components built into the 
program design to address many of these challenges. For example, in 
response to safety concerns, an urban grantee elected to provide 
transportation for youth attending evening events. However, grantees and 
others said more planning time would have been beneficial. 

Grantees and others reported that the youth and their communities made 
progress toward the YO program goals, but the program’s impact is still 
under study. Grantees reported that they had enrolled about 91,000 youth 
nationwide, many of whom completed high school, entered college, or found 
employment after enrolling in the program. In addition, grantees and others 
said that the grants had benefited their communities. However, without an 
impact analysis, it is not known whether these events would have occurred 
in the absence of the program. The Department contracted for a $24 million 
evaluation of the program that included plans for an impact analysis; 
however, agency officials are unsure if the analysis will be completed. 

Manchester Point Arena Youth Opportunity Center, Calif. 
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Accountability * Integrity * Reliability 



United States Government Accountability Office 
Washington, DC 20548 



December 9, 2005 

The Honorable Arlen Specter 
Chairman 

Subcommittee on Labor, Health and Human Services, 

Education, and Related Agencies 
Committee on Appropriations 
United States Senate 

Dear Mr. Chairman: 

More than 5 million youth between the ages of 16 and 24 were jobless and 
out of school in 2001, according to a recent study, 1 and young people living 
in high poverty areas face particularly difficult barriers to employment and 
education. A variety of factors make this population hard to serve, 
including low levels of academic attainment, limited work experience, and 
a scarcity of jobs in their communities. Much remains to be learned about 
the best ways to help these youth connect with education and employment 
opportunities. A 2003 report by the White House Task Force for 
Disadvantaged Youth concluded that a lack of rigorously collected 
program evaluation data has limited the potential for gaining insight into 
the strategies that are most effective for serving these youth. 

One program that explored innovative strategies was the Youth 
Opportunity Grant program, authorized under the Workforce Investment 
Act of 1998. 2 This was a 5-year, $1 billion program aimed at increasing the 
educational attainment and long-term employment of youth residing in 
impoverished areas. The program was the successor to several prior 
demonstration programs that were based on a similar model. Through the 
Youth Opportunity Grants awarded under the program, the Department of 
Labor targeted funds to 36 localities with high poverty rates and employed 
a comprehensive approach for attempting to improve opportunities for at- 
risk youth living in these communities. All 14 to 21 year-olds residing in 
the target areas, regardless of income, were eligible to receive services, 



1 Andrew Sum, Ishwar Khatiwada, Nathan Pond, and Mykhaylo Trub’skyy, Left Behind in 
the Labor Market: Labor Market Problems of the Nation’s Out of School, Young Adult 
Populations. Center for Labor Market Studies (Northeastern University: November 2002). 

2 Pub. Law. No. 105-220. 
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and once enrolled, could usually remain in the program as long as needed. 
The program design included many of the strategies experts have 
suggested are effective in serving at-risk youth, including providing a safe 
place for young people to go, assigning a key adult to monitor their 
progress, and offering a wide array of services. The program also provided 
a system for grantees to report performance information and included 
plans for an extensive evaluation study. The program began in 2000 and 
was funded for 5 years. 

In order to understand what can be learned from the Youth Opportunity 
Grant program, you asked us to examine the following: (1) How did 
grantees implement the Youth Opportunity Grant program? (2) What 
challenges did they face, and what can be learned from their experiences 
in addressing them? (3) What is known about the outcomes and impact of 
this program, both on the participants and on their communities? 

To answer these questions, we surveyed the 36 Youth Opportunity Grant 
program directors, visited 7 grantees, and analyzed program data provided 
by the Department of Labor. All of the program directors completed the 
survey, which we conducted between March and May 2005. 3 We conducted 
five in-depth site visits to grantees, during which we interviewed program 
administrators, staff, youth, and other members of the community. We also 
toured youth opportunity centers, in-school facilities, and observed 
program activities. We selected these five sites according to the 
proportionate representation of urban, rural, and Native American 
grantees as well as for their geographic distribution. We briefly visited two 
additional grantees while pretesting the survey, and at these sites, we 
toured grantees’ centers and spoke with program administrators, staff, and 
youth. In addition, we reviewed agency documents and interviewed 
agency officials and relevant experts. To address the third question 
regarding outcomes and impact, we also analyzed electronic data the 
Department of Labor collected from the grantees in a management 
information system developed specifically for the program. 

We conducted our work from September 2004 through November 2006 in 
accordance with generally accepted government auditing standards. 



3 To view selected results from the survey, go to GAO-06-56SP. 
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Results in Brief 



Grantees carried out the Youth Opportunity Grant program — which was 
designed in part to enhance the local infrastructure of youth services — in a 
variety of ways. As outlined in the program design, grantees set up centers 
in the target areas that served as focal points for the program and housed a 
core staff of trained youth workers who provided case management and 
individualized services to participants. Because the target areas varied in 
size, some grantees had 1 center while other, more geographically- 
dispersed grantees had as many as 40. Grantees also offered participants a 
range of youth services in the areas of education, occupational training, 
leadership development, and support services. Youth could participate in 
an array of activities in these areas, from college tours and subsidized 
employment to community service projects and health-related workshops. 
Grantees collaborated with other organizations to deliver some of these 
services, such as arranging for youth to attend GED preparation classes at 
a local agency. In many cases, grantees also developed unique 
programming to fill gaps in available services. One grantee started a 
charter high school, the only one of its kind in that city. All of the grantees 
tracked the activities of participants and transmitted the information to a 
reporting system administered by the Department of Labor. In addition, 
many grantees used innovative methods to recruit participants, such as 
placing outreach staff in malls and providing work-related incentives for 
youth to participate. 

Grantees faced challenges related to program start-up, conditions in their 
communities, and meeting the service needs of the youth; however, certain 
features of the program design such as local discretion and individualized 
services, helped grantees address these challenges. Most of the grantees 
reported that it was difficult to set up a center within the 7 months the 
Department of Labor expected the programs to be operational and 
grantees said that this quick start-up had an adverse effect on retaining 
participants. As a consequence, grantees, experts, and agency officials said 
that programs could have benefited from additional time to plan. In 
addition, although most grantees found it difficult to establish an 
information-reporting system, some of them ultimately found it useful for 
their own program management purposes. Most grantees reported that 
existing conditions in their communities were problems — such as 
violence, drugs, and lack of jobs — but grantees used the discretion 
afforded to them by the program to devise a range of approaches to 
address the problems. For example, a rural grantee responded to limited 
employment opportunities in its area by subsidizing positions with local 
businesses and providing the youth transportation to jobs in other locales. 
Additionally, grantees told us that the obstacles facing their clients — such 
as low academic achievement, health problems, and lack of family 
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support — made it difficult for the youth to move forward, but found 
aspects of the program’s design — such as case management and 
individualized services — useful in helping the youth deal with these 
barriers. For example, one grantee trained case managers to conduct 
mental health assessments of the youth and, when deemed appropriate, 
arranged for a professional counselor to provide therapy at the center. 

Grantees and others reported that the participants and their communities 
made progress toward the education and employment goals of the 
program; however, a formal assessment of the program’s impact is still 
under study. Data provided by the grantees show that the Youth 
Opportunity Grant program enrolled about 91,000 youth nationwide, a 
number of whom subsequently completed high school, entered college, or 
found employment. In addition, grantees and others told us that the 
program benefited their communities. However, in the absence of a 
systematic impact analysis, it is not known whether the events they cited 
would have occurred without the program. The Department of Labor 
contracted for a $24 million evaluation of the program that included plans 
for an impact analysis. However, agency officials are unsure if the impact 
analysis will be completed as originally planned due to the Department’s 
allocations being less than expected. 

In order to learn from this innovative program and continue to improve 
efforts to serve at-risk youth, we recommend the Secretary of Labor take 
the actions necessary to complete the impact analysis of the Youth 
Opportunity Grant program and release the impact data and all related 
research reports from the program’s evaluation. In comments on GAO’s 
draft report, the Department of Labor agreed with our recommendation 
and indicated that it plans to complete the impact analysis and publish all 
related reports. 



Background 



The Workforce Investment Act of 1998 authorized Youth Opportunity 
Grants (YO), a $1 billion, 5-year program aimed at increasing the 
educational attainment and long-term employment of youth residing in 
high poverty areas. The program was designed to assist at-risk, hard-to- 
serve youth and the communities in which they live by concentrating 
resources into a targeted geographic area. YO expanded on earlier 
demonstration programs that were based on a similar design, the most 
recent of which were known as Kulick grants. The Department of Labor 
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(the Department) announced the 36 YO grantees 4 — 24 urban, 6 rural, and 6 
Native American — in February 2000 and set a goal for the grantees to have 
their programs operational by September of that year. Although the 
Department originally planned to continue to add grantees, funding for the 
program was eliminated in the budget for fiscal year 2004. Figure 1 shows 
the 36 grantees by location. 



4 For the purposes of this report, we refer to the entity that operated the program as a 
grantee. The 36 recipients of the grants were workforce investment boards, states, 
counties, cities, and other entities. Some recipients contracted out the operations of the 
program, while others directly offered services to youth. 
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Figure 1: Location of Youth Opportunity Grantees 



Portland, 

Ore. 




_ Imperial 
Los Angeles,# County, 
Calif \ Calif, f 
San Diego, Calif# l~| 



Oglala 

Sioux 



Denver, Colo. 



Grand Traverse 
~V, Band of 
Ottawa and 
OChippewa Indians 



Milwaukee, # Buffalo. N Y. 

Wise# Detroit. 

V ) I Mich# 

\ / — ^#^ \ Philadelphia... 

Cleveland, Penn. # 
Ohio 



Oute Mountain 



Navajo 

Nation O 



Kansas 
• City. 
Mo. 



Southeast 

Arkansas/ 



^tuisville, 
Ky. , 



^Memphis. Tenn. 



Robeson ■ 
County, N.C.-S> 



Cook Inlet 
Tribal Council 

o ^ 



Island of Moloka'i, 
4° Hawaii 

C Q, 



San Antonio, 
Tex. “ 



Arkansas # 

|~l ^ Birmingham, \/ 

g 1 Ala. ) Albany, Gay 

' ; Northeast —A c( 

Houston, Louisiana •, y, L 

Tex. ( ofeT \ 

* T n L 

, Tampa. Fla# 



lyL Brockton. 
\ Mass. 

^ Hartford, 
Conn. 

Baltimore, Md. 
Washington, D.C. 



Youth Opportunity Grantees 
• Urban (24) 

□ Rural (6) 

O Native American (6) 



Source: GAO analysis of Department of Labor information. 



Note: In Robeson County, N.C., and Imperial County, Calif., the program only operated in certain 
areas of the county. 



Page 6 



GAO-06-53 Youth Opportunity Grants 






The Department of Labor’s stated goals for the YO program were to 
increase educational attainment and promote long-term employment of 
youth in the target areas, as well as to improve the youth service delivery 
systems in these communities. Specifically, grantees were expected to 
effect increases in the rates of high school completion, college enrollment, 
and employment, both for youth participating in the program and, by 
extension, the overall target area. Grantees were also expected to facilitate 
the delivery of services by partnering with other institutions in the 
community, especially local schools and colleges, the juvenile justice 
system, and private employers. The program was designed to allow 
grantees to enhance the local infrastructure of youth services by filling in 
critical service gaps, coordinating existing services, and, in the case of 
some rural and tribal areas, developing an infrastructure where services 
were limited. 

Many of the YO program components were based on practices that experts 
have identified as effective for working with at-risk youth, and the design 
of the program supported flexibility and invention at the local level. Based 
on a youth development framework that emphasizes a comprehensive 
approach to meeting a young person’s needs, the program required YO 
grantees to offer a full range of education, employment, and leadership 
development services as well as provide other supports to the youth. 
Advocates and experts have also stressed that youth need a place to gather 
where they feel comfortable and a sense of belonging. In this vein, YO 
grantees were required to have at least one youth center in the target 
community that would provide a focal point for services and activities as 
well as a safe place for the youth to go. In addition, grantees were 
expected to maintain a core staff of trained youth workers and to provide 
follow up services to participants for at least 2 years after they completed 
participation in program activities, which is in keeping with the identified 
effective practice of providing youth access to a caring, trusted adult for 
an extended period of time. Additionally, experts have suggested youth 
benefit from the opportunity to continue a relationship with a program for 
as long as they need. Similarly, the program had flexible enrollment rules 
that allowed participants to remain enrolled even through periods of 
inactivity. However, the program design allowed grantees latitude in 
deciding many of the particulars of the program. For example, grantees 
could choose how and by whom services would be delivered, the number 
and location of the YO centers, and the institutions with which they 
partnered. Figure 2 displays the key components of Youth Opportunity 
Grant program. 



Page 7 



GAO-06-53 Youth Opportunity Grants 




Figure 2: Key Components of Youth Opportunity Grant Program 
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Source: GAO analysis of Workforce Investment Act of 1998 and Department of Labor information. 
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Like some prior demonstration programs that the Department of Labor has 
administered, eligibility for YO was not based on income, but rather on 
geographic residence. Youth between the ages of 14 and 21 living in the 
target area were eligible for the program, regardless of income. In most 
cases, the target areas were federally designated empowerment zones or 
enterprise communities, which are, by definition, areas with high rates of 
poverty. 

The Department of Labor provided extensive, ongoing technical assistance 
to grantees. The Department assigned grantees coaches with considerable 
experience working in youth programs who helped grantees with a range 
of issues, such as developing services and building partnerships. The 
Department also collaborated with a private foundation to establish a 
training institute that provided courses in youth development to more than 
2,000 youth workers. In addition, the Department sponsored opportunities 
for grantees to share best practices and strategies in regular directors’ 
meetings and conference calls and through peer-to-peer training sessions. 

The Department laid the foundation for collecting performance data on 
the program by setting up a management information system (MIS). The 
Department collected performance data on the YO program in a 
centralized, electronic system. Grantees submitted information to fulfill 
requirements of the Workforce Investment Act, which mandated that 
grantees report on seven performance measures related to employment, 
retention, earnings, attainment of skills, and attainment of credentials. In 
addition to these, grantees reported on interim measures developed by the 
Department to gauge the progress of participants as they moved through 
the program. These measures were designed to document youth 
participation in activities and other intermediate milestones. 

The Department had funded several prior programs premised on a 
geographic and community concept similar to that of the YO, although on 
a much smaller scale, but little information is available on the impact of 
these demonstration programs. Although evaluations were conducted of 
these earlier programs, the impact studies were either incomplete or not 
released. The first of these demonstrations, Youth Opportunities 
Unlimited, was evaluated and the results published, but the study did not 
include a systematic analysis of impact based on comparison groups. The 
funding for the second demonstration, Youth Fair Chance, was cut 2 years 
into the program, and although the evaluation study was published, it was 
based on only 2 of the planned 5 years of the program. As a result, the 
authors of this study advised that the findings should be interpreted 
cautiously. The most recent of the demonstration programs was known as 
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the Kulick grants. The Department has prepared an evaluation of the 
Kulick grants, but has not yet released the results publicly. 

For the Youth Opportunity Grants, the Department set up an extensive 
effort in 2000 to collect and analyze information to assess the program’s 
impact. The Department contracted for a $24 million evaluation study that 
included plans to estimate the impact of the program by comparing key 
characteristics in the YO communities and comparable areas that did not 
receive YO grants. It was to begin in 2000, and the final report was 
originally scheduled to be completed in July 2005. The study was designed 
to differentiate between those observed changes in participants and the 
communities that resulted from the program and those that would have 
occurred even without the program’s intervention. 



Grantees Carried Out 
the Program Using a 
Variety of Approaches 



Grantees adapted key elements of the program’s design to their particular 
circumstances and used a variety of approaches in providing services and 
conducting outreach. Grantees set up centers that varied widely in terms 
of number and other characteristics where youth could receive services 
and participate in activities. The centers housed a core staff who provided 
case management and helped plan individualized services for youth. Staff 
numbers and duties differed between grantees. Most grantees used the 
management information system (MIS) provided by the Department of 
Labor to submit program data to the Department, but some used their own 
systems. Most grantees used a combination of approaches to provide 
youth services, including collaborating with other organizations and, when 
they deemed it necessary, developing services of their own. To reach hard- 
to-serve target population, grantees used a variety of recruiting techniques, 
ranging from the conventional to the innovative. 



Grantees Adapted The centers set up under the terms of the grant varied widely in number 

Program’s Design to Their ami character. Because the target communities varied in size, some 

Particular Circumstances grantees had 1 center while other, more geographically dispersed grantees 

had as many as 40. The centers were intended to serve as the hub of the 
local programs, and the types of activities offered in them differed 
considerably. One center we visited in rural Louisiana had a variety of 
recreational facilities, such as a basketball court and recreation room, in 
addition to a classroom, computer lab, and staff offices. Youth 
participating in this program told us nothing like these facilities had 
existed for them before the YO program. On the other hand, a center we 
visited in Houston resembled a traditional career center, with computer 
kiosks set up near staff offices where the youth could undertake job 
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searches and skills assessments. Houston officials said they had formerly 
offered some recreational activities on site, but found them to be 
duplicative of other services available in the neighborhood. Some grantees 
provided still other facilities at their centers. For example, Baltimore had 
an on-site health clinic and music recording studio. Some grantees also 
had centers located within schools. The District of Columbia had staff 
sited in a local high school, where they offered training and support for 
youth. This in-school center was located in a large classroom, with a staff 
office off to one side and furnished with a computer lab with equipment 
purchased by the grantee. Figure 3 depicts photos of two youth 
opportunity grant centers we visited. 



Page 11 



GAO-06-53 Youth Opportunity Grants 




Figure 3: Photos of YO Centers 




REACH-Milwaukee Youth Center 




Louisiana East Carroll YO! Center 



Source: GAO. 



The centers housed staff who varied substantially in number and makeup. 
Houston divided staff duties between as many as 115 people working in 4 
centers. In this program, “personal service representatives” provided case 
management and identified youths’ goals and training needs, and 
“employment counselors” assisted youth with job searches and conducted 
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follow-up services to youth who had been placed in employment. Other 
grantees, however, had sites with only one person performing all of the 
major staff duties. At the California Indian Manpower Consortium (CIMC) 
Manchester Point Arena site, one youth worker handled recruitment, case 
management, and job placement. Because of the remoteness of the site, 
this youth worker also frequently drove a van, weekly transporting youth 
to services as far as 4 hours away. 

Most grantees used the management information system (MIS) provided 
by the Department of Labor to record program data, but some used their 
own systems and then submitted the data to the Department. Those who 
did not use the Department MIS used a variety of systems. For example, 
Milwaukee developed its own system with the aim of using it in 
conjunction with other youth programs offered by its local Workforce 
Investment Board. 5 Similarly, Hartford developed customized software 
that enabled it to share information with other provider agencies and the 
schools in the community. 



Grantees Used 
Collaboration and 
Invention to Varying 
Degrees in Delivering 
Services 



Grantees used a combination of approaches to establish a network of 
educational, occupational, and other services for youth, but varied the 
extent to which they relied on other providers. Grantees availed 
themselves of existing services, either through formal arrangements or by 
referring participants to other organizations in the community. Kansas City 
arranged for youth to attend classes at other local agencies to help prepare 
for the GED exam. Similarly, Baltimore referred young women who 
needed interview clothing to a local charity organization. Most grantees 
also collaborated with other institutions to provide some services. San 
Antonio partnered with a local community college to establish academies 
in which youth could spend half a day on their home campuses and half a 
day at the community college, allowing them to earn a certificate or 
associate degree in areas such as aviation or biotechnology. Houston 
purchased credit retrieval software for local high schools that students 
could use to complete their academic requirements for graduation. In 
addition, grantees originated programs to fill perceived gaps in services. 
Memphis established its own charter school with an emphasis on the arts 
in order to provide alternative education opportunities for the youth, one 
of the emphasized youth activities in the program. To provide leadership 



5 As stated in the Workforce Investment Act, a local Workforce Investment Board is an 
entity designated to set policy for the statewide workforce system within the local area. 
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development opportunities, one of the CIMC sites sponsored a cemetery 
care project. In this project, participants worked with the tribal elders to 
make a map of the local cemetery with the names of the deceased because 
cemeteries on this reservation did not have headstones. Table 1 shows 
youth activities and examples of how grantees implemented the activities. 



Table 1 : YO Program Areas, Corresponding Youth Activities, and Examples of Implementation 



YO program area 


Youth activities 


Example of implementation 


Improving 

educational 

achievement 


Tutoring, study skills 
training, and instruction, 
including dropout 
prevention strategies 


Birmingham created an intensive 9-week dropout prevention program called 
Operation Yes that targeted 8th, 9th, and 10th graders functioning one or more grade 
levels below their current grade. It combined academic remediation with job 
readiness training, life skills sessions, and a paid work experience. 




Alternative secondary 
school services 


Cleveland partnered with their local school district to offer on-site the Twilight School 
program, a diploma-track high school for dropouts. The program offered mastery- 
based credit classes at a time and in a setting conducive to these youth. 


Preparation for and 
success in 
employment 


Summer employment 
opportunities directly 
linked to academic and 
occupational learning 


Louisville partnered with the local bar association to provide summer internships for 
interested YO participants. 




Paid and unpaid work 
experiences, including 
internships and job 
shadowing 


Boston developed the Transitional Employment Services program, a four-tiered 
approach to employment training and job placement. Youth could progress from the 
lowest tier, which focused on basic employability skills and gaining experience 
through community service projects, to the highest level, which focused on helping 
youth find unsubsidized employment. 




Occupational skills 
training 


Moloka’i partnered with a local fishpond to give youth hands-on experience in areas 
such as basic computing, seaweed farming, aquaculture, carpentry, and plumbing. 


Services to develop 
the potential of 
youth as citizens 
and leaders 


Leadership development 
opportunities 


Brockton trained youth to be mediators and youth coordinators to resolve conflicts at 
the YO center and in their community. The program was funded through a grant from 
the Attorney General’s office. 


Supports for youth 


Supportive services 


Cook Inlet Tribal Council created the Summer Bridging Institute, a program that 
brought youth from small villages who had been accepted to the University of Alaska, 
Anchorage to campus before the start of the fall semester. The program offered youth 
the chance to acclimate to urban campus life and to bond with other students, with 
the goal of increasing the likelihood they would stay in college. 




Follow up services for at 
least 24 months 


Houston switched from following up quarterly to following up monthly with youth who 
had been placed in employment. Staff found it was easier to keep track of the youth if 
they contacted them more frequently. 




Adult mentoring 


Southeast Arkansas assigned adult mentors to youth according to similar interests or 
career goals. Adults spent at least 2 hours per week with the youth and reported their 
activities to the program through written documentation. 




Comprehensive guidance 
and counseling 


Baltimore arranged for a community organization to hold weekly support groups for 
young fathers. A service specialist presented a parenting education curriculum, with 
group sessions intended to improve the involvement of fathers in the lives of their 
children. 



Source: Workforce Investment Act of 1998 and GAO analysis of documents from Department and grantees. 
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Grantees Actively 
Recruited Youth, Including 
the Hard to Serve, and 
Used Incentives to Draw 
and Maintain Involvement 



Grantees also used an array of strategies to help youth find long-term 
educational or occupational opportunities and to follow up with 
participants after they exited the program. Some grantees had staff 
specifically devoted to finding appropriate employment opportunities and 
connecting youth with these opportunities. For example, Milwaukee 
created the position of job developer, who identified potential employers 
and spoke to them directly about the youth in the program. Grantees also 
used a number of methods for helping youth enroll in post-secondary 
institutions, such as conducting college tours, assisting with financial aid 
forms, and placing support staff on college campuses. The statutory 
provision authorizing the YO program required grantees to provide follow- 
up services for 2 years after the youth completed participation in program 
activities. Grantees recorded on a quarterly basis the employment and 
education status of the youth who had completed participation in the 
program, for example, if they were in school or working at the time of 
contact. Some grantees contacted youth more frequently. Staff in 
Baltimore told us they varied the frequency depending on the degree to 
which they regarded the young person to be at some risk. One might be 
contacted monthly, while another would be checked on nearly every day. 

Grantees recruited youth, including traditionally difficult to serve 
populations, by creating connections with other youth-serving agencies. 
Milwaukee operated a juvenile justice project to help youth coming out of 
the corrections system to reintegrate into the community. Personnel at a 
local correctional facility worked with program staff to identify those due 
for release who were expecting to live within the geographic target area of 
the program. Program staff traveled to the facility to conduct an 
orientation session and then worked with the youth, corrections officials, 
and parents to develop a reintegration plan. The services offered to them 
were similar to those offered to all of the YO participants. 

Grantees also used a variety of innovative methods to recruit participants. 
Since many of the youth were disconnected from both school and work, as 
one grantee told us, they could be not only hard to serve, but hard to find. 
Some grantees went beyond conventional outreach activities of mailers 
and radio advertisements and conducted community walking campaigns 
using staff to saturate shopping malls and other areas where youth 
congregate. Others used youth to lead recruitment. In some cases, a 
grantee used employment as an inducement to link youth to other 
activities the program offered. For example, Louisiana’s work program 
allowed participants to work more hours in subsidized employment the 
further they advanced toward their educational goals. We were told many 
of the youth lived in unstable economic conditions, and these needs had to 



Page 15 



GAO-06-53 Youth Opportunity Grants 




be addressed before the youth would focus on other areas, such as 
education. In other cases, grantees used cash or non-monetary rewards to 
encourage participation. In Houston, youth could attend special events, 
such as a basketball game, if they participated in the program a minimum 
number of hours. 



Grantees Used 
Program Design to 
Address Major 
Challenges, but Found 
Start-up Time to be 
Short 



Grantees were able to address a variety of challenges in setting up the 
program and delivering services to youth using local discretion, flexible 
enrollment rules, and other aspects of the program’s design, but they and 
others said a longer start-up period would have been beneficial. Most 
grantees found it difficult to establish centers and retain participants in 
their programs. They felt that these challenges were compounded by 
expectations for a quick start-up, and subsequently they and Department 
officials felt more planning time would have been beneficial. Grantees also 
had difficulty establishing an information-reporting system, but once in 
place, found it was helpful for program management purposes. Conditions 
in the communities such as violence and lack of jobs presented a 
challenge to most grantees, but they took advantage of the local discretion 
built into the program to develop strategies to address them. Grantees also 
cited as an obstacle the vast service needs of many of the youth the 
program was intended to serve, but case management, individualized 
services, and flexible enrollment rules were useful in dealing with them. 



Grantees Felt Challenged 
to Set Up Program in Time 
Allotted, and More 
Planning Time Would Have 
Been Beneficial 



The majority of grantees reported that finding or renovating centers was a 
challenge. Many grantees did not have suitable sites in their communities 
for use as centers, and some put considerable effort into renovating 
existing structures. In the extreme, the grantee in Alaska renovated 40 
centers in remote communities not accessible by road and shipped 
equipment and material by air. In a few cases, grantees did not have a 
permanent building until after the first year of the program. In Boston, the 
program was housed in temporary facilities for a period of time because a 
permanent center could not be completed for the program’s opening. 



Another challenge identified by most grantees was retaining participants, 
which grantees linked in part to expectations from the Department to start 
to enroll youth quickly. The Department set a goal to recruit 3,000 
participants nationwide within the first few months of the program, or a 
little less than 100 youth per grantee. Some grantees said they felt rushed 
to meet recruitment goals and told us they recruited many youth who were 
not committed to the program. In addition, several directors mentioned 
that they felt they were asked to serve participants before their 
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programming was fully developed, a situation that as one expert 
commented, was akin to asking grantees to “fly the plane before they were 
finished building it.” We were told that, in these situations, some youth 
became disenchanted with the program and left because of the limited 
offerings the center presented early on. 

Grantees, agency officials, and experts said a longer planning period 
would have been beneficial. Grantees had an 7 month window between 
the announcement of the grant recipients and the Department’s target date 
for having the programs operational. For example, two grantees with 
whom we spoke said about three additional months would have been 
necessary for them to meet the goals set out by the Department. Agency 
officials also told us that grantees were pushed to start-up too quickly and 
could have benefited from more time to plan. Agency officials said it 
would have been appropriate to give grantees more time to meet the 
Department’s initial goals. 



Grantees Had Difficulties 
Establishing an 
Information Reporting 
System, but Found It 
Useful Once in Place 



Most grantees reported that establishing an information reporting system 
was challenging, and grantees told us it would have been useful for the 
Department to have had a workable system in place at the outset of the 
program. The management information system (MIS) initially provided by 
the Department was fraught with problems. Department officials told us it 
took about 18 months to iron out the kinks. In addition, the Office of 
Inspector General identified inconsistencies in the way grantees were 
recording data early on. 6 Although the Department eventually developed 
an improved system, grantees said it would have been better had the MIS 
been in place from the beginning. Not having a system in place created 
additional burden for the grantees. For example, Louisiana developed its 
own system to collect information until an updated system was available 
and then manually transferred 11 months of data into the new MIS. 



Once in place, grantees additionally used the information system for their 
own program management purposes such as to improve performance and 
reinforce accountability. The Hartford grantee was able to integrate its 
system with the school district, allowing case managers to track 
attendance and grades to allow better monitoring of participant 
achievements. The Hartford system also enabled program management to 



6 Office of Inspector General, Department of Labor, Workforce Investment. Act, Youth 
Opportunity Program Audit, OIG Audit Report Number 06-03-001-03-390 (March 2003). 



Page 17 



GAO-06-53 Youth Opportunity Grants 





see which case managers were most successful at engaging youth in the 
activities required for the completion of their goals. Regarding 
accountability, Portland used its information system to monitor the 
performance of its contractors on a weekly basis. 



Grantees Used Their 
Discretion to Address 
Community-Based 
Challenges 



Some of the major challenges also identified by grantees were problems 
external to the program, yet affected their ability to deliver services. 
Twenty-eight of the grantees reported on the survey that a lack of jobs in 
the community was a challenge. In some areas, jobs were on the decline 
because of shifts in the local economy or relocation of major employers. 
For example, the grantee in San Francisco related that due to the dot-com 
bust, youth in the program were competing with a skilled workforce 
willing to fill entry level positions that would otherwise be available to 
youth. In other areas, especially rural ones, there were few employers, let 
alone large ones. In addition, 29 of 36 grantees reported on the survey that 
risk factors such as violence, drugs, and gangs were challenges in 
implementing the program. In some cases, these factors made it difficult 
for participants to receive services. In one urban community, we were told 
there were safety concerns with youth participating in evening activities 
sponsored by the YO because they would have to return home after dark. 

Local discretion built into the program design helped grantees respond to 
external challenges. Within the structure of the program, grantees were 
allowed some amount of latitude to develop responses to circumstances in 
their communities, such as a scarcity of jobs. For example, the California 
Indian Manpower Consortium created opportunities for youth to have 
work experiences, such as subsidizing summer jobs with local business 
including a campground and senior center or working with the tribe to 
place youth in clerical positions. Similarly, grantees were able to develop 
services to address specific risk factors in their communities. Imperial 
County offered a curriculum series to help address interpersonal violence 
among adolescents, an issue they had identified as particularly 
problematic in their area. The series was designed to reduce impulsive and 
aggressive behavior through empathy training, interpersonal problem 
solving, behavior skill training, and anger management. In Milwaukee, the 
grantee addressed safety concerns by renting vans to transport the youth 
to nighttime events. 
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Service Needs of 
Participants Presented 
Challenges, but Tailored 
Services, Individualized 
Planning, and Flexible 
Enrollment Rules Were 
Useful in Addressing Them 



Grantees identified as a major challenge the obstacles faced by their 
clients, such as homelessness, lack of family support, mental health 
problems, and low levels of academic attainment. Staff in Baltimore told 
us homelessness was a frequent issue faced by youth in their area and 
finding a place to sleep can preoccupy the youth and disrupt the learning 
process. YO staff and others said that participants may lack support in 
other areas of their lives, such as from their families. For example, one 
grantee told us of a participant who was awarded a full scholarship to 
college, but the parents would not sign the paperwork to receive the 
money. Some youth also faced mental health issues, which prevented them 
from moving forward in their lives. All of these factors, we were told, 
could mean a slow start for youth who enrolled in the program. The 
director of an alternative high school in Milwaukee told us that because of 
the disorder in their lives, many of these youth may take a year just to 
become comfortable in the program before they can even begin making 
any forward progress. 

Grantees found aspects of the program design such as case management, 
individualized services, and flexible enrollment rules useful in addressing 
the service challenges of participants. For example, grantees used 
assessments to help determine the academic needs of clients and provide 
each client with the appropriate individualized services. Philadelphia 
designed a program in which each participant had a personalized 
remediation plan based on the results of an evaluation test and interview 
with an education coach. This plan was intended to build on clients’ 
strengths and accounted for their particular learning styles. After the 
initial assessment, participants were reevaluated at regular intervals to 
monitor their academic progress and goal attainment. The curriculum was 
designed so that youth could increase at least one grade level after 90 
hours of instruction. In addition, both youth and community members told 
us that YO staff were key in helping youth stay motivated and guiding 
them through difficult situations. Grantees also found the enrollment rules 
useful in working toward program goals. The program’s enrollment policy 
allowed most participants to continue to receive services even if they 
experienced periods of inactivity. Staff in Houston told us that the policy 
was a benefit to the youth because, unlike other employment programs, 

YO did not end their relationship with youth who had not been 
participating for a while. 
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Grantees and Others 
Reported That 
Participants and Their 
Communities Made 
Progress, but the 
Program’s Impact Is 
Still under Study 



Grantees and others reported that the participants and their communities 
made advancements in education and employment; however, a formal 
assessment of the program’s impact is still under study. Data reported by 
the grantees showed that a number of youth advanced their education 
while they were participating in the program, such as completing high 
school. Grantees also reported data showing a portion of the participants 
entered unsubsidized employment after enrolling in the program. 

Similarly, grantees and others believe their communities made 
advancements toward the Department’s stated education and employment 
goals of the program. In addition, the majority of grantees reported that 
they made improvements in the youth service delivery systems in their 
communities. The Department funded an evaluation to assess the impact 
of the program, which was designed to shed light on the extent to which 
observed changes can be attributed to the program. However, the study 
has not yet been completed. 



Grantees Reported Data 
Showing Some 
Participants Advanced 
Their Education and 
Found Employment 



Data reported by the grantees showed that a number of youth advanced 
their education while they were participating in the program. Two of the 
Department’s primary goals of program were to increase the rates of high 
school completion and college enrollment for the youth. Data that 
grantees entered into the management information system (MIS) showed 
that of the approximately 91,000 youth who signed up for in the program 
nationwide, about 18,700 either completed high school or attained a GED 
after enrolling in the program. In addition, about 11,700 youth entered 
college, 37 percent of whom were reported to be out of school when they 
initially enrolled in the program. Youth with whom we spoke credited the 
program with giving them a second chance to increase their education and 
to better their employment opportunities. One Louisiana youth’s 
comments typified what we heard elsewhere from participants during our 
site visits, “YO is the best thing that has happened to me. YO has given me 
a job and put a little money in my pocket.” He added, “YO has also been 
instrumental in keeping me out of jail. Above all, YO has helped me re ali ze 
that education is important.” Figure 4 summarizes enrollment, 
participation, education, and employment data for youth in the program, 
as of June 2005. 
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Figure 4: Number of Youth Who Signed up for, Participated in, and Achieved at Least One Educational or Employment Goal of 
the Program 




Educational status after Long-term placements 

enrollment in program 

Source: GAO analysis of YO MIS data through June 2005. 

Note: The five educational status and long-term placement categories are not mutually exclusive. 



Grantees reported data showing that a portion of the youth entered 
unsubsidized employment after completing YO activities. Data from the 
MIS show that about 17,300 youth were placed in employment that was 
not subsidized by the program, 62 percent of whom were out of school 
when they initially enrolled in the program. Grantees and others told us 
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the job readiness training that some youth received as part of the program 
was particularly important in helping them get jobs. Employers we spoke 
with told us they knew the youth that were sent to them by YO would be 
well trained and ready to work. A local human resources manager for a 
national chain of home improvement stores told us that he thought YO 
participants were better prepared for employment than other residents of 
the area, adding that they show up with better job skills and “soft” skills. 



Grantees and Others 
Believe Their Communities 
Moved toward Program 
Goals 



The majority of grantees reported making improvements in the youth 
service delivery systems in their communities. Some of the described 
changes were to the service infrastructure in these communities. For 
example, San Diego pointed out that it had created a multiservice youth 
center in a neighborhood where none had existed before. Grantees cited 
other improvements related to the mode of service delivery. Tucson 
reported that prior to YO, it had funded three stand-alone youth programs, 
but now it has a one-stop system with multiple entry points for delivering 
youth services. Other grantees noted that their efforts to foster better 
communication and collaboration among service providers had benefited 
the youth. As the Portland grantee commented, by partnering with their 
local employment department and community college system, they were 
able to leverage staff to provide intensive job search and college 
preparation services to youth. In other cases, local leaders credited the YO 
as a catalyst for change in their community. For example, a school 
superintendent in California told us that YO staff had helped to forge an 
agreement between the school district and five Indian tribes in the area to 
develop an education curriculum that would be more sensitive to the 
tribes’ cultures. 



Grantees and others expressed the view that their communities had made 
advancements in concert with the Department’s education and 
employment goals for the program. Almost all of the grantees reported 
that high school completion, college enrollment, and youth employment 
rates improved in their communities as a result of the program. Several 
grantees also asserted that affecting the youth led to changes in the 
community. In addition, community members with whom we spoke said 
the program helped their communities make progress toward greater 
education and employment. For example, a tribal leader in California told 
us that YO motivated the youth in his tribe to stay in school. Similarly, one 
expert pointed to some of the rural grantees who had many youth enroll in 
college, which she said was especially important because of a lack of jobs 
in these areas. 
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Department of Labor’s 
Study of Program Impact 
Has Not Been Completed 



While the Department of Labor funded an evaluation designed to assess 
the impact of the program, it has not yet been completed. This was 
planned as a 5-year, $24 million evaluation, which began in 2000. As part of 
the evaluation, the Department conducted baseline and follow-up surveys 
of the target areas, and gathered extensive descriptions of the 36 programs 
and communities. The evaluation also included plans for an impact study 
that was designed to compare YO participants and communities with 
other, similar youth and communities that did not participate in YO. This 
type of comparative analysis would be necessary to determine if the 
events reported by grantees and others would have occurred in the 
absence of the program. However, the evaluation is not finished. 

Moreover, agency officials are unsure if the impact study that was to be 
part of the evaluation will be completed. Due to departmental allocations 
being lower than expected, the Department spent $1.9 million less than the 
full amount of the original contract on the evaluation. Agency officials told 
us that the impact study was likely to be scaled back because of the lower 
allocation. The evaluation was originally scheduled to be completed in 
July 2005. However, agency officials told us they do not expect the study 
to be finished until June 2006. 



Conclusions 



The Youth Opportunity Grant program was designed to help at-risk youth 
and their communities by concentrating resources geographically and by 
incorporating components that experts have suggested are effective in 
assisting this population. To continue to improve the ability to serve these 
youth, researchers and practitioners must be able to learn from the most 
promising and innovative approaches in serving these youth, including 
those used in the Youth Opportunity Grant program. The Department of 
Labor has begun an evaluation of the program that includes many pieces 
of a potentially useful study. Although informative, these pieces will not by 
themselves answer the question of impact, in other words, whether the 
described events would have occurred in the absence of the program. In 
order to understand what effect, if any, the program had on these 
observed events, it is necessary to have a systematic comparison with 
other, similar communities that did not receive grants. The Department 
planned, but has not yet completed, such an analysis as part of the 
evaluation. The Department has an opportunity to contribute to the 
research on programs for serving at-risk youth, if the evaluation study is 
completed and the results made public. However, the Department has not 
taken full advantage of past opportunities to release information on other 
programs based on a similar model, such as the Kulick grants. Given the $1 
billion investment in this program — including almost $24 million for an 
evaluation effort — and the need for rigorous data on these types of 
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programs, the study should be completed and the results should be made 
available. Unless the Department completes the evaluation of the Youth 
Opportunity Grant program and releases the results, researchers and 
practitioners will not be able to fully realize the potential to learn from the 
program. 



Recommendation for 
Executive Action 



To continue to improve efforts to serve at-risk youth and in order that 
researchers can evaluate the quality of information and determine possible 
impact of the program, we recommend that the Secretary of Labor take 
the actions necessary to complete the impact analysis of the Youth 
Opportunity Grant program and release the data and all related research 
reports from the program’s evaluation. 



Agency Comments 



We provided a draft of this report to the Department of Labor for its 
review and comment. In its response, the Department agreed with our 
conclusions and recommendation and indicated that it intends to complete 
the impact analysis and publish all related reports from the Youth 
Opportunity Grant program evaluation. A copy of the Department’s 
response is in appendix II. The Department also provided us with 
technical comments, which we incorporated into the report where 
appropriate. 



We will send copies of this report to the Secretary of Labor, relevant 
congressional committees, and other interested parties. We will also make 
copies available to others upon request. In addition, the report will be 
available at no charge on GAO’s Web site at http://www.gao.gov. 
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Please contact me at (415) 904-2272 if you or your staff have any questions 
about this report. Contact points for our Offices of Congressional 
Relations and Public Affairs may be found on the last page of this report. 
Key contributors to this report are listed in appendix III. 

Sincerely yours, 




David D. Beilis 

Director, Education, Workforce, and 
Income Security Issues 
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Appendix I: Objectives, Scope, and 
Methodology 



To accomplish our research objectives, we surveyed Youth Opportunity 
Grant Program directors on the implementation of the Youth Opportunity 
Grants. To augment information from our survey, we conducted five site 
visits of programs. We chose sites for our visits to represent 
proportionately the three grantee types (urban, rural, and Native 
American) as well as their geographic distribution. The grantees we visited 
were the District of Columbia, Houston, Milwaukee, rural Louisiana, and 
the California Indian Manpower Consortium. During our visits we met 
with program administrators, staff, and participants to learn their 
perspectives on program implementation, challenges, outcomes, impact, 
and lessons learned. Also, we interviewed local workforce board 
members, school officials, community based service providers, private 
employers, and local government officials to discuss their perspectives on 
the impact and other aspects of the program. In addition, we toured Youth 
Opportunity Centers, in-school facilities, and observed program activities. 
We briefly visited an additional two grantees — Baltimore and 
Philadelphia — in the course of pretesting the survey, and while at their 
sites, toured centers and spoke with program administrators, staff, and 
participants. We gathered additional information by reviewing agency 
documents including site assessments and grant applications. We also 
interviewed agency officials and other relevant experts, including 
researchers, representatives of advocacy organizations, YO coaches, and 
former Department of Labor officials. We performed our work from 
September 2004 to November 2005 in accordance with generally accepted 
government auditing standards. 

We also investigated several possible sources of external data to 
quantitatively measure outcomes and impact, but determined none of 
them were feasible for our purposes. The data sets we reviewed were: 
Current Population Survey, Common Core of Data, National Longitudinal 
Survey of Youth, High School and Beyond, and American Community 
Survey. 



Survey Of Youth To learn about the implementation of the Youth Opportunity Grants, we 

Opportunity Grant conducted a Web-based survey of ah Youth Opportunity Grant Program 

Program Directors Directors. We asked directors about the usefulness of program 

components, challenges they faced implementing the program and 
strategies used to deal with them, and their opinion on the program’s 
impact on participants and communities. Additionally, we asked directors 
about the size and structure of their programs, the manner in which they 
delivered services, and the types of organizations with which their 
programs partnered. The survey also included a series of questions about 
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Appendix I: Objectives, Scope, and 
Methodology 



how programs maintained information to help us determine the reliability 
of data in the Department of Labor’s management information system. We 
pretested the survey with several program directors and modified the 
survey to take their comments into account. All 36 program directors 
completed the survey, for a response rate of 100 percent. We administered 
the survey between March 17 and May 31, 2005. To view selected results of 
the survey, go to GAO-06-56SP. 



Department of Labor 
Management Information 
System (MIS) 



We used electronic data collected on the program by the Department of 
Labor in a management information system (MIS) to describe the number 
of youth in the program who achieved the Department’s goals for the 
program, such as high school completion, college enrollment, and long- 
term education and employment placements. We have determined the MIS 
data are sufficiently reliable for the purposes of this study through 
discussions with officials at the Department of Labor, in-depth interviews 
with MIS staff during site visits, and responses to a comprehensive array 
of survey items in which each grantee described their procedures for 
editing and auditing the data they entered into the MIS. We analyzed the 
MIS data using guidance provided to us by the Department. 
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Appendix II: Comments from the Department 
of Labor 



U.S. Department Of Labor Assistant Secretary tor 

Employment and Training 
Washington, D.C. 20210 

NOV 2 2 2005 

Mr. David D. Beilis 
Director, Education, Workforce, and 
Income Security Issues 
U.S. Government Accountability Office 
441 G Street, NW 
Washington, DC 20548 

Dear Mr. Beilis: 

Thank you for providing the draft report GAO-06-53 entitled, “Youth Opportunity Grants: 
Lessons Can Be Learned from Program, but Labor Needs to Make Data Available.” We 
appreciate your efforts in identifying innovative grantee practices and lessons learned that we 
may consider fer future grant initiatives. 

We have learned many important lessons through the Youth Opportunity Grant (YOG) program. 
First, and most importantly, the YOG program demonstrated that money alone does not solve the 
problems faced by at-risk youth. Many local sites struggled for a number of years because of a 
lack of clear vision and solid local leadership. Some locations suffered from a lack of clear focus 
on what is most important for youth: educational attainment and meaningful job and career 
opportunities. Based on our experience, the most successful youth programs are ones that utilize 
a demand-driven approach whereby private sector funds are leveraged and invested, quality 
secondary and post-secondary programs are accessible and local leadership is outcome-oriented 
and dynamic. 

In response to the recommendation to complete the evaluation of the YOG program, the U.S. 
Department of Labor (DOL) always has been unequivocally committed to completing the 
analysis. Unfortunately, some of the initial evaluation design was flawed and focused on 
"perception” and process rather than impact, but we have done our best to refocus the design to 
get to meaningful analysis of impact and outcomes. 

Based on recent discussions between DOL and the evaluation contractor, we do not believe at 
this time that completing the analysis will require adding funds beyond what has already been 
obligated to the evaluation contract. DOL will release the impact data and all related reports 
from the program’s evaluation, including the Kulick demonstration reports, when completed. 

Enclosed are DOL’s technical comments on the draft report. If you would like additional 
information, please do not hesitate to call me at (202) 693-2700. 
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